1. Field of the Invention
The present invention generally relates to image processing and, more particularly, to an image capturing apparatus and method for selective real-time focus and/or other parameter adjustment, hereinafter referred to as focus/parameter adjustment.
2. Description of the Related Art
An image capturing apparatus, such as a digital still camera, digital video camera, a camera contained within a portable communication terminal (e.g., a cell phone), and/or the like, generally includes some mechanism for autofocus (AF) which performs focal position detection at each of a plurality of focus detection points in a field. The AF function typically performs control to focus on a focus detection point that is automatically or intentionally selected from among the focus detection points.
A focal point is a point where light rays originating from a point on an object of a scene converge, and the focal point is associated with sharpness perception. Adjustment of the focal point is one of the most important aspects in photography. The focal point is not strictly speaking just a point but has a spatial component because of aberrations and diffractions in the imaging optics that can be accentuated by increasing aperture and field of view.
Early AF was active AF based on an ultrasonic reflected sound emitter and detection of the echo. This evolved to the use of infrared signals using triangulation, amount of reflected sound emitter, and detection of the echo. FIG. 4 illustrates this conventional approach. In this approach, an AF sensor, typically an infrared sensor, estimates the distance. Based on parameters of the optics (lens) and the AF mode, typically centered focus or focus on other part of the image, the AF sensor estimates the focal position that is appropriate for the lens.
In step S401, data from an AF sensor is obtained. In step S403, optics parameters obtained from step S405 and a pre-programmed focus mode obtained from step S407 are used for estimation of focusing position based on pre-programming. A one shot capture focus adjustment is made in step S409. Some advanced digital still cameras, such as the Canon EOS 5D Mark II, provide a limited number of regions to select for AF in the manual mode. However, AF does not provide the user with a natural way to select particular regions of the scene with different levels of focusing.
AF has evolved to encompass the preservation of a wider range of focal points relative to the scene being captured. There are methods to combine multiple images with a range of focus conditions, such as described in the Malzbender patent (U.S. Pat. No. 6,741,251 B2, issued on May 25, 2004). More recently, with the advent of the field of computational photography, there are methods to capture four dimensional light fields in specific image encoding and post-process the images in order to compute rendering of images in distinct focal points, such as the method proposed in the Ng application (U.S. Patent Application Publication No. 2008/0131019 A1, filed on Nov. 30, 2007).
The Ng application approach is illustrated in FIG. 5, where an image is captured in step S502 with a camera 500 by using a multi-aperture lens or sensor in step S504, and the captured data is processed after being transferred to a computing system, such as a server 510. The user can determine the focal plane in step S512 and the computer renders the appropriate image in step S516 using image refocusing software in step S514, resulting in focused image S516. Currently, the computational photography method described in the Ng application can only be accomplished by post-processing captured images and there is no feedback to the user while capturing the images. The Ng application also does not allow the user to creatively and selectively choose degrees of focusing and defocusing on one or more parts of the image during capture of the image.
Although preserving light fields is beneficial to re-render images under distinct focal points, these approaches have several drawbacks. (1) Current approaches are computationally intensive and require memory storage and therefore these approaches rely on post-processing in computers after the images are captured.
(2) There is currently no way for the photographer to specify, during photo shooting, which regions of a scene the photographer wants to control focusing and which degree of sharpness. Focusing is made automatically with AF. The alternative for the photographer is to shoot multiple pictures with manual focus and combine them resulting in mis-registration problems.
(3) Creative art photographers do not want every object in the scene to be focused. (4) Current AF methods do not work for all focusing and defocusing needs in creative photography.
Manual focusing generally occurs without any camera user interface aid. AF is generally provided without any real-time selectivity for degrees of focusing and defocusing for one or more parts of a scene. Computational photography multi-aperture generally occurs with post-processing without any feedback during capture on how the image would look like in the rendering.
Therefore, a need exists for an image capturing apparatus and method for image region selective real-time focus/parameter adjustment.